Optimal Control of Markov Processes

The purpose of this article is to giye an overview of some recent developments in optimal stochastic control theory. The field has expanded a great deal during the last 20 years. It is not possible in this overview to go deeply into any topic, and a number of interesting topics have been omitted entirely. The list of references includes several books, conference proceedings and survey articles. The development of stochastic control theory has depended on parallel advances in the theory of stochastic processes and on certain topics iti partial differential equations. On the probabilistic side one can mention decomposition and representation theorems for semimartingales, formulas for absolutely continuous change of j)robability measure (e.g. the Girsanov formula), and the study of Ito-sense stochastic differential equations with discontinuous coefficients. It seems fair to say that these developments in stochastic processes were in turn to an extent influenced by their applications in stochastic control. For controlled Markov diffusion processes, there is a direct connection with certain nonlinear partial differential equations via the dynamic programming equation. These equations are of second order, elliptic or parabolic, and possibly degenerate. Stochastic control gives a way to represent their solutions probabilistically. There is an unforeseen connection with differential geometry via the Monge-Ampère equation. Broadly speaking, stochastic control theory deals with models of systems whose evolution is affected both by certain random influences